Redundancy-Related Bounds on Generalized Huffman Codes

This paper presents new lower and upper bounds for the compression rate of optimal binary prefix codes on memoryless sources according to various nonlinear codeword length objectives. Like the most well-known redundancy bounds for minimum (arithmetic) average redundancy coding — Huffman coding — these are in terms of a form of entropy and/or the probability of the most probable input symbol. The bounds here improve on known bounds of the form L ∈ [H,H + 1), where H is some form of entropy in bits (or, in the case of redundancy measurements, 0) and L is the length objective, also in bits. The objectives explored here include exponential-average length, maximum pointwise redundancy, and exponentialaverage pointwise redundancy (also called dth exponential redundancy). These relate to queueing and single-shot communications, Shannon coding and universal modeling (worst-case minimax redundancy), and bridging the maximum pointwise redundancy problem with Huffman coding, respectively. A generalized form of Huffman coding known to find optimal codes for these objectives helps yield these bounds, some of which are tight. Related properties to such bounds, also explored here, are the necessary and sufficient conditions for the shortest codeword being a specific length.